49 research outputs found
Probabilistic Error Analysis for Inner Products
Probabilistic models are proposed for bounding the forward error in the
numerically computed inner product (dot product, scalar product) between of two
real -vectors. We derive probabilistic perturbation bounds, as well as
probabilistic roundoff error bounds for the sequential accumulation of the
inner product. These bounds are non-asymptotic, explicit, and make minimal
assumptions on perturbations and roundoffs.
The perturbations are represented as independent, bounded, zero-mean random
variables, and the probabilistic perturbation bound is based on Azuma's
inequality. The roundoffs are also represented as bounded, zero-mean random
variables. The first probabilistic bound assumes that the roundoffs are
independent, while the second one does not. For the latter, we construct a
Martingale that mirrors the sequential order of computations.
Numerical experiments confirm that our bounds are more informative, often by
several orders of magnitude, than traditional deterministic bounds -- even for
small vector dimensions~ and very stringent success probabilities. In
particular the probabilistic roundoff error bounds are functions of
rather than~, thus giving a quantitative confirmation of Wilkinson's
intuition. The paper concludes with a critical assessment of the probabilistic
approach
Conditioning of Leverage Scores and Computation by QR Decomposition
The leverage scores of a full-column rank matrix A are the squared row norms
of any orthonormal basis for range(A). We show that corresponding leverage
scores of two matrices A and A + \Delta A are close in the relative sense, if
they have large magnitude and if all principal angles between the column spaces
of A and A + \Delta A are small. We also show three classes of bounds that are
based on perturbation results of QR decompositions. They demonstrate that
relative differences between individual leverage scores strongly depend on the
particular type of perturbation \Delta A. The bounds imply that the relative
accuracy of an individual leverage score depends on: its magnitude and the
two-norm condition of A, if \Delta A is a general perturbation; the two-norm
condition number of A, if \Delta A is a perturbation with the same norm-wise
row-scaling as A; (to first order) neither condition number nor leverage score
magnitude, if \Delta A is a component-wise row-scaled perturbation. Numerical
experiments confirm the qualitative and quantitative accuracy of our bounds.Comment: This version has been accepted to SIMAX but has not yet gone through
copy editin
Probabilistic Linear Solvers: A Unifying View
Several recent works have developed a new, probabilistic interpretation for
numerical algorithms solving linear systems in which the solution is inferred
in a Bayesian framework, either directly or by inferring the unknown action of
the matrix inverse. These approaches have typically focused on replicating the
behavior of the conjugate gradient method as a prototypical iterative method.
In this work surprisingly general conditions for equivalence of these disparate
methods are presented. We also describe connections between probabilistic
linear solvers and projection methods for linear systems, providing a
probabilistic interpretation of a far more general class of iterative methods.
In particular, this provides such an interpretation of the generalised minimum
residual method. A probabilistic view of preconditioning is also introduced.
These developments unify the literature on probabilistic linear solvers, and
provide foundational connections to the literature on iterative solvers for
linear systems